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Abstract.  An  efficient  algorithm  is  presented  for  communicating  letter-shape  information  from 
a  high-speed  computer  with  a  large  memory  to  a  typesetting  device  that  has  a  limited  memory. 
The  encoding  is  optimum,  in  the  sense  that  the  total  time  for  typesetting  is  minimised,  using  a 
model  that  generalizes  well-known  “demand  paging”  strategics  to  the  case  where  changes  to  the 
cache  are  allowed  before  the  associated  information  is  actually  needed.  Extensive  empirical  data 
shows  that  good  results  are  obtained  even  when  difficult  technical  material  is  being  typeset  on  a 
machine  that  can  store  information  concerning  only  100  characters.  The  methods  of  this  paper 
are  also  applicable  to  other  hardware  and  software  caching  applications  with  restricted  lookahead.^ 
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1.  Introduction. 

The  purpose  of  this  paper  is  to  study  a  data- reduction  problem  that  arises  when  computers 
are  applied  to  phototypesetting.  A  page  that  is  printed  with  modern  typesetting  equipment  may 
be  regarded  as  a  gigantic  matrix  of  0’s  and  l’s,  where  0  represents  a  blank  Bpace  and  1  represents 
ink.  For  example,  the  particular  machine  used  in  our  experiments  has  approximately  19,000,000 
such  bits  per  square  inch;  therefore  a  typical  page  of  technical  text  from  a  book  like  [4],  which  was 
printed  on  that  machine,  is  essentially  a  matrix  of  more  than  727  million  bits.  This  data  must  be 
reduced  by  more  than  three  orders  of  magnitude  in  order  to  be  transmitted  from  the  host  computer 
to  the  typesetter  at  a  rate  of  6600  bits  per  second,  if  the  page  is  to  be  finished  in  less  than  two 
minutes.  Typical  methods  of  data  compression  are  considered  excellent  if  they  achieve  a  reduction 
factor  of  only  50  per  cent,  so  it  is  clear  that  special  techniques  are  needed  if  high- resolution  digital 
printing  is  to  be  efficient. 

The  main  factor  accounting  for  this  thousand-fold  reduction  in  the  number  of  information 
bits  is,  of  course,  the  fact  that  pages  are  composed  from  letters  that  have  comparatively  simple 
shapes.  For  example,  a  typical  character  that  measures  5  X  10  printer’s  points,  where  there  are 
72.27  printer’s  points  per  inch,  has  a  digital  pattern  occupying  about  182,000  bits  on  the  machine 
mentioned  above,  but  this  pattern  can  be  specified  satisfactorily  with  about  250  bytes  =  2000 
bits. 

Even  with  this  reduction,  however,  there  remain  about  2500  characters  per  page,  so  about 
5,000,000  bits  still  need  to  be  transmitted.  The  problem  would  be  simple  if  the  typesetter  knew 
all  of  the  digital  patterns  for  all  of  the  letters,  since  we  would  merely  have  to  transmit  letter 
codes.  But  typical  technical  texts  involve  a  variety  of  different  fonts  and  special  symbols,  and 
many  typesetting  machines  have  only  a  limited  local  memory  for  the  storage  of  character  patterns. 
Therefore  the  character  shapes  must  be  transmitted  from  the  host  computer  to  the  typesetter,  and 
the  only  way  we  can  compress  this  data  is  by  using  the  fact  that  most  characters  arc  used  again 
and  again. 

For  example,  suppose  that  the  typesetter  has  enough  memory  to  record  the  shapes  of  60 
characters.  This  is  just  barely  enough  for  the  letters  a  to  *  and  A  to  Z,  but  we  also  need  to  deal 
with  numerals  and  punctuation  marks,  together  with  italic  and  bold  variations,  and  with  changes 
in  site  and  style.  The  standard  industrial  practice  has  been  to  solve  the  sise-change  problem 
by  doing  simple  scaling  operations,  so  that  "8-point  type”  is  obtained  as  an  80%  reduction  of 
“10-point  type”;  but  typographers  are  very  unhappy  about  this  compromise,  because  the  results 
were  much  better  on  the  old  hot-lead  machines  when  every  point  sise  was  designed  separately. 
Fortunately  it  turns  out  that  individual  lines  of  text  hardly  ever  need  a  great  variety  of  characters 
even  without  the  compromise;  therefore  the  typesetter  can  use  its  memory  as  a  “cache”  for  60 
characters,  including  the  30  or  so  that  it  needs  on  the  current  line. 

The  typesetter  might  also  be  able  to  accept  a  few  more  character  descriptions  that  will  be 
needed  on  subsequent  lines,  at  the  same  time  as  it  is  setting  type  on  the  current  line;  these  new 
characters  can  replace  “dead”  ones  in  the  cache,  and  with  luck  the  cache  will  be  up  to  date  at  all 
times.  For  example,  if  there  is  time  to  make  live  adjustments  to  the  cache  on  each  line,  30  new 
characters  can  be  brought  in  when  a  new  font  is  desired,  if  the  changes  begin  six  lines  in  advance. 
By  looking  ahead  to  see  which  characters  need  to  be  sent  in  the  future,  the  host  computer  can 


control  the  typesetter’s  cache  contents  in  an  efficient  way.  The  purpose  or  this  paper  is  to  examine 
suitable  algorithms  by  which  the  host  computer  can  exhibit  such  clairvoyant  behavior,  and  to 
study  how  much  is  gained  by  such  techniques. 

Section  2  presents  a  theoretical  model  of  a  general  cache  allocation  problem,  and  Section  3 
derives  an  optimal  allocation  strategy  for  that  model.  Data  structures  and  algorithms  by  which  the 
optimum  strategy  can  be  computed  with  reasonable  efficiency  are  described  in  Sections  4  and  5. 
The  concluding  section  presents  empirical  results  that  illustrate  what  can  be  achieved. 

Although  this  paper  is  oriented  towards  a  particular  application  to  typesetting,  the  reader 
is  encouraged  to  speculate  about  how  the  same  methods  could  be  applied  to  the  design  of  ultra- 
high-speed  computers.  One  can  imagine  a  pipelined  arithmetic  unit,  playing  a  role  analogous  to 
that  of  the  typesetter,  taking  orders  from  another  computer,  whose  function  is  to  preload  a  cache 
memory  with  numeric  data,  based  on  the  knowledge  of  a  particular  algorithm's  control  structure. 
Instead  of  relying  on  the  conventional  architecture  of  a  general  purpose  computer,  one  could  apply 
the  methods  of  this  paper  to  a  large  class  of  important  computation-intensive  algorithms  whose 
control  structure  is  predictable. 

2.  A  cache- allocation  model. 

Consider  an  alphabet  of  m  possible  characters  that  might  be  kept  in  a  cache  that  can  hold  at 
most  a  characters  at  once.  We  wish  to  implement  a  sequence  of  commands  of  the  following  three 
types: 

L(t)  Lock  character  i  in  the  cache,  where  1  <  t  <  m. 

U(i)  Unlock  character  i. 

G  Get  any  character  and  place  it  into  the  cache. 

Such  a  sequence  is  called  a  “job.”  Character  »  is  said  to  be  “wedged”  at  a  certain  point  of  a  job 
if  more  L(i)  commands  than  l/(i)  commands  have  occurred  before  that  point.  Wc  assume  that 
the  U(i)  command  appears  only  when  character  t  is  wedged,  so  at  any  point  in  reading  through  a 
job,  we  will  never  have  seen  more  L(i)  commands  than  U(i)  commands  for  any  *.  Furthermore  we 
assume  that  there  are  never  more  than  s  different  characters  wedged  at  any  one  time;  a  job  that 
does  not  meet  this  requirement  needs  a  larger  cache. 

Initially  the  cache  has  s  empty  slots.  When  an  L(t)  command  occurs  and  character  t  is  not 
present  in  the  cache,  we  say  that  there  is  a  “fault.”  In  the  ease  of  a  fault,  the  typesetter  comes 
to  a  halt  while  character  *  is  brought  into  the  cache,  either  going  into  an  empty  slot  or  replacing 
some  unwedged  character.  Similarly,  at  the  time  of  a  G  command,  any  character  not  present  in 
the  cache  can  be  brought  into  a  cache  slot  that  is  not  occupied  by  a  wedged  character.  In  this 
case,  we  do  not  consider  that  a  fault  has  occurred  since  the  typesetter  is  still  busy  doing  a  previous 
line.  Thus,  G  commands  allow  us  to  anticipate  L  commands  so  that  future  faults  are  avoided.  It 
is  also  possible  to  "pass”  a  G  command,  leaving  the  cache  unchanged,  If  this  seems  more  desirable 
than  bringing  in  a  new  character. 

Note  that  a  character  must  be  In  the  cacho  whenever  it  is  wedged,  because  an  £(*')•  command 
guarantees  that  character  i  la  present,  and  because  no  character  can  be  replaced  until  it  has  become 
unwedged. 


This  model  is  more  general  than  the  “page  reference"  model  that  is  usually  used  to  study 
cache  behavior  in  a  virtual  memory  system.  The  page  reference  model  is  the  special  case  in  which 
there  are  no  G  commands,  and  where  each  Lfa  command  is  immediately  followed  by  U(i).  Our 
model  also  assumes  that  we  know  the  entire  sequence  of  commands  in  a  job  before  the  job  is  begun. 

In  our  application,  the  typesetting  of  a  full  line  must  operate  in  real  time  with  no  waiting 
for  faults.  Thus,  a  line  of  type  containing  the  characters  *1*2 . . . might  be  represented  by  the 
command  sequence 

L(it)  L(ia) . . .  L(ir)  Gk  Ufa)  Ufa) . . .  Ufa) , 

if  there  is  time  to  bring  into  the  cache  as  many  as  k  characters  for  future  lines  while  the  line  is  being 
typeset.  The  actual  typesetting  of  the  line  starts  after  the  command  Lfa)  has  been  completed. 
This  sequence  of  commands  ensures  that  all  characters  needed  on  the  line  will  be  present  in  the 
cache  before  typesetting  takes  place.  The  L  instructions  for  line  (i  4  1)  are  not  begun  until  the 
typesetter  has  completed  line  t,  so  that  no  characters  needed  on  the  line  will  get  over-written 
before  the  typesetter  is  done  with  them. 

The  model  does  not  specify  what  character  is  replaced  at  the  time  of  a  fault  or  of  a  G 
command.  A  “caching  strategy”  w  a  set  of  rules  that  govern  what  happens  to  the  cache  at  such 
times.  A  “strategy  trace”  is  the  output  of  a  strategy  when  it  is  fed  a  job;  in  other  words,  it  is  a 
list  that  records,  at  each  G  and  L  command,  which  character,  if  any,  is  to  be  brought  into  which 
cache  slot. 


3.  An  optimum  catching  strategy. 

In  this  section  we  shall  see  that  an  intuitively  plausible  strategy  for  cache  allocation  actually 
minimises  the  total  number  of  faults,  among  all  possible  strategics  for  a  given  command  sequence. 
(This  generalises  Belady’B  well  known  “MIN”  method  in  the  page  reference  model  (1,2,5].) 

The  strategy  is  simply  this: 

1)  Whenever  a  character  is  brought  into  the  cache,  place  it  in  an  empty  slot,  if  possible; 
otherwise  let  it  replace  an  unwedged  character  t  that  never  appears  in  »  subsequent  Lfa 
command,  if  possible;  otherwise  let  it  replace  the  unwedged  character  »  in  the  cache  whose 
next  appearance  in  an  Lfa  commend  is  as  late  as  possible.  Since  at  most  s  characters 
can  be  wedged  at  once,  one  of  these  three  cases  must  always  hold. 

2)  Whenever  a  G  command  appears,  bring  in  the  character  »  not  currently  in  the  cache, 
whose  next  appearance  in  an  lfa  command  is  as  soon  as  possible,  unless  all  unwedged 
characters  currently  in  the  cache  will  be  locked  by  L  commands  that  occur  between  the 
current  G  command  and  this  lfa  command,  or  unless  no  such  character  *  exists. 

When  a  character  is  brought  into  the  cache  by  rule  (3),  its  cache  slot  is  selected  by  to  rule  (1). 
The  ease  in  rale  (2)  where  no  such  i  exists  occurs  when  all  the  chacters  needed  by  the  rest  of  the 
job  are  already  in  the  cache. 


To  prove  that  this  strategy  S  is  optimum,  we  shall  compare  its  trace  on  any  job  to  any  other 
possible  trace  for  the  same  job,  and  show  that  5's  trace  leads  to  no  more  faults.  More  precisely, 
let  Sj  be  5’s  trace  for  any  job  J.  If  Xj  is  different  from  Sj,  we  shall  construct  a  trace  X'j 
that  has  no  more  faults  than  Xj  at  any  time,  and  X  j  agrees  with  Sj  longer  than  Xj  does.  In 
other  words,  if  Xj  agrees  with  Sj  on  the  first  1  —  1  commands  but  differs  from  it  at  command 
number  t,  then  Xj  will  agree  with  Sj  for  at  least  the  first  t  commands.  Repeated  application  of 
this  argument  will  show  that  S  leads  to  the  smallest  possible  accumulated  number  of  faults  at  all 
times. 

The  construction  we  shall  define  makes  use  of  a  “trace  completion  subroutine”.  The  input  to 
the  trace  completion  subroutine  consists  of  a  job  J,  a  trace  Wj  that  implements  J,  and  a  partial 
trace  Yj  that  is  only  defined  through  the  { p-  l)th  command  of  J.  The  subroutine  will  complete 
the  definition  of  Yj,  such  that  it  is  at  least  as  good  as  Wj.  The  following  conditions  must  hold 
just  after  command  (p—  1)  for  both  Wj  and  Yj  (omitting  the  implied  subscript  *  J *): 

i)  There  are  characters  w  and  y  such  that  the  cache  for  W  has  the  form  {to}  U  C  and  the  cache 
for  Y  has  the  form  {y}  U  C,  for  some  set  of  characters  C,  where  to  j t  C  and  y  Jf  C.  In  other 
words,  the  caches  are  identical  except  for  at  most  one  element. 

ii)  Trace  W  has  had  at  least  as  many  faults  as  trace  Y . 

iii)  If  the  sequence  of  future  commands  causes  to  to  be  locked  before  y,  where  to  and  y  are  the 
characters  mentioned  in  condition  (i),  then  W  has  already  had  more  faults  than  Y . 

The  second  condition  says  that  Y  is  no  worse  than  W.  The  third  condition  says  in  effect  that 
character  to  cannot  be  a  “better”  thing  to  have  in  the  cache  than  y,  unless  Y  can  afford  one  more 
fault  without  falling  behind  W. 

If  these  three  conditions  are  satisfied,  we  shall  say  that  “relation  (to,  y)  holds  for  (IV,  Y,J)n  at 
the  current  position  in  the  job.  The  trace  subroutine  is  called  only  when  relation  (w,y)  holds  for 
(W,Y,J)  at  command  p—  1.  The  subroutine  proceeds  by  figuring  out  what  Y  should  do  for  the 
pth  command  in  order  to  preserve  these  invariant  conditions.  In  other  words,  if  relation  (to,  y) 
holds  before  the  pth  command  of  J,  the  subroutine  shall  define  the  next  step  of  Y  so  that 
relation  (to'.y')  holds  after  the  pth  command,  for  some  to'  and  y*.  The  subroutine  can  now  do 
the  (p+  l)th  command,  and  so  on,  until  Y  has  been  defined  for  all  of  J.  Since  the  invariants  still 
hold,  we  know  by  (ii)  that  Y  is  no  worse  than  W,  so  the  subroutine  docs  what  was  claimed. 

Now  to  define  Y  on  the  pth  command.  Wo  know  that  relation  (te,y)  holds.  If  to  =  y,  bo  that 
both  traces  currently  have  the  same  cache  contents  according  to  condition  (i),  we  simply  let  Y  be 
the  same  as  W  on  command  p.  Relation  (is,  y)  still  holds.  (In  this  case,  the  next  iteration  of  the 
subroutine  will  have  w  =  y,  so  the  (p  +  l)th  action  of  Y  will  again  be  defined  to  be  the  same  as 
that  of  W  by  this  rule,  and  so  on,  so  from  this  time  henceforth  Y  is  the  same  as  W.) 

On  the  other  hand  if  w  jd  y,  note  that  both  w  and  y  must  currently  be  unwedged,  since  w 
does  not  oeeur  in  Vs  cache  and  y  does  not  appear  in  W’a.  The  following  subcases  arise  in  defining 
Y  on  command  |; 


a)  If  the  command  iB  L(y),  so  that  a  fault  occurs  with  trace  W,  suppose  W  replaces  z  by  y. 
Trace  Y  has  no  fault,  so  it  can’t  bring  a  character  into  the  cache;  but  after  the  command 
L(y),  it  is  easy  to  check  that  relation  (tv,  z)  holds,  because  condition  (ii)  implies  that  W  has 
now  had  more  fa.ults  than  Y. 

b)  If  the  command  is  L(w),  so  that  a  fault  occurs  with  Y  but  not  W,  then  Y  replaces  y  in 
its  cache  by  w.  This  replacement  is  legitimate,  because  y  is  currently  unwedged.  Afterwards 
relation  (to,  to)  holds,  since  condition  (iii)  implies  that  this  case  can  arise  only  if  Y  could  afford 
at  least  one  fault. 

c)  If  the  command  is  L(i)  where  i  to,  t  5^  y,  and  t  g  C,  a  fault  occurs  in  both  traces.  If  W 
replaces  to  by  *,  then  Y  replaces  y  by  relation  (t,t)  holds.  Otherwise,  if  w  replaces  z  by  t 
for  some  z£C,  then  Y  also  replaces  z  by  t,  and  relation  (to,y)  still  holds. 

d)  If  the  command  is  f>(»),  where  i  €  C,  or  if  it  is  U[i),  no  fault  occurs  for  cither  W  or  Y ,  and 
relation  (to,  y)  remains  true. 

e)  U  the  command  is  G  and  if  W  replaces  to  by  v,  then  Y  replaces  y  by  v,  and  relation  (v,  v) 
holds. 

f)  Finally,  if  the  command  is  G  and  if  W  replaces  z  by  t>  for  some  z  €  C,  or  if  W  does  nothing, 
then  Y  likewise  replaces  z  by  v  or  does  nothing.  Relation  (to,  y)  still  holds. 

This  completes  the  definition  of  Y  from  W,  except  in  one  degenerate  case:  Suppose  that  the 
command  is  L(y)  and  that  W  brings  character  y  into  an  empty  position  in  its  cache.  This  is  a 
variation  on  case  (a),  where  Y  cannot  braing  in  a  character  because  no  fault  has  occurred.  We  can 
avoid  this  situation  by  assuming  that  the  set  C  in  condition  (i)  always  contains  s  —  1  elements, 
i.e.,  that  there  are  no  empty  positions.  For  we  can  1111  each  empty  position  with  distinct  dummy 
characters  that  do  not  appear  in  any  commands;  this  convention  makes  the  proof  go  through. 

Now  that  the  trace  subroutine  has  been  specified,  we  shall  use  it  to  prove  the  optimality  of 
strategy  S.  Suppose  Xj  is  any  trace  different  from  Sj  for  some  job  J .  The  first  difference  occurs 
at  the  tth  command  in  the  traces.  We  will  create  a  trace  X'j  to  be  the  same  as  Sj  up  to  and 
including  the  tth  command,  such  that  relation  (z,  y)  holds  for  (Xj,  X'j,  J),  for  some  x  and  y.  We 
can  then  call  the  trace  subroutine  to  complete  Xj  such  that  It  is  at  least  as  good  as  Xj.  Then 
we  will  be  auie  to  repeat  the  process  with  Sj  and  X'j,  getting  X",  which  is  like  Sj  through  the 
(i  4-  l)th  command  and  at  least  as  good  as  X'j  (and  therefore  at  least  as  good  as  Xj),  and  so  on. 
The  final  result  is  that  Sj  is  the  same  as  Jf  for  some  n  <  lcngth(J),  and  Sj  is  no  worse  than 
Xj.  Since  J  and  X  are  arbitrary,  this  will  prove  that  5  is  optimal.  (Once  again,  we  mil  drop 
the  J  when  it  is  understood.) 

80  the  only  task  left  is  to  show  that  if  X'  is  defined  to  be  the  same  as  S  through  command  t, 
then  relation  (*,**)  holds  for  (X,  X',J)  for  some  s  and  s'.  Just  before  command  t,  both  X  and 
X'  have  had  the  same  number  of  faults,  and  both  their  caches  have  the  same  contents.  The  tth 
command  must  either  be  aa  L  command  that  canaos  a  fault,  or  a  G  command  on  which  8  and  X 


tmm 


didn’t,  both  pass.  Suppose  first  that  command  t  is  L(i),  where  t  is  not  in  either  cache  at  time  t,  and 
X  replaces  character  j  by  t  while  S  replaces  character  Jb  by  *.  Relation  (k,  j)  holds  Tor  (X,X',J) 
because  rule  (1)  guarantees  that  character  k  is  not  locked  before  character  j . 

Similarly,  if  the  tth  command  is  G,  and  if  X  passes  while  S  replaces  k  by  z,  relation  (Jfc,s) 
holds  for  (X,X \J)  since  rules  (1)  and  (2)  imply  that  k  is  not  locked  before  z.  And  if  X  replaces 
j  by  to  when  implementing  a  G  command,  while  S  passes  on  that  G,  relation  (to,.;)  holds,  since 
rule  (2)  ensures  that  to  is  not  locked  before  j. 

The  only  remaining  case  is  that  the  tth  command  is  G,  and  that  trace  X  replaces  j  by  z 
while  5  replaces  k  by  to.  If  j  =  k,  relation  (a,  to)  holds  for  (Jf, X',  J)  because  of  rule  (2).  On  the 
other  hand,  if  j  k,  we  have  to  invoke  the  trace  subroutine  twice  before  obtaining  a  trace  that 
dominates  X  and  agrees  with  5  on  commands  1  through  t :  we  first  let  Z  be  a  trace  that  replaces  k 
by  z,  so  that  Z  is  a  mixture  of  X  and  S.  At  this  point,  relation  (k,j)  holds  for  (S,  Z,J),  because 
of  rule  (1).  Completing  Z  with  the  trace  subroutine,  we  now  have  a  trace  that  is  still  different 
from  S  in  the  tth  command.  This  command,  however,  is  a  G  command  where  Z  replaces  k  by  z, 
while  X'  replaces  k  by  to,  and  so  relation  (z,  to)  holds  for  (Z,X',J). 

Wc  have  now  shown  that  it  is  always  possible  to  set  up  Xj  to  obey  the  invariant  conditions, 
and  this  finally  completes  the  proof  that  S  is  optimum. 

4.  Implementing  the  optimum  strategy. 

Let  m  be  the-total  number  of  possible  characters,  let  s  be  the  size  of  the  cache,  and  let  n  be 
the  number  of  commands  in  job  J.  Our  goal  is  to  have  an  algorithm  that  computes  the  optimal 
trace  Sj.  Job  J’ a  commands  are  in  arrays  op  and  char  before  the  algorithm  begins.  If  the  jth 
command  is  L( »),  U(i),  or  G,  then 

op  [7]  =  lL\  char  [7]  =  t, 

or  op(j]  =  ‘U’,  char  [7]  =  », 

or  op  (j)  =  ‘G\  char  [7]  =  undefined, 

respectively,  for  1  <  7  <  n.  For  the  present  we  shall  pretend  that  we  have  enough  memory  to 
store  all  n  of  the  commands  at  once. 

The  algorithm  records  the  resulting  trace  in  the  cache  and  char  arrays.  If  cache\j\  >  0, 
step  j  of  the  trace  says  to  bring  character  char\j\  into  cache  slot  cache\j]\  and  when  cachefj )  =  0, 
then  no  character  is  to  be  brought  into  the  cache  during  step  7.  Thus,  if  a  fault  occurs  at  the 
7U1  command,  the  algorithm  should  set  cache  [7*)  to  the  cache  position  that  5  allocates  to  char[j], 
where  1  <  cache  [7]  <  1.  If  op[j]  =  ‘G’  and  if  strategy  S  replaces  cache  position  k  by  character  c, 
the  algorithm  should  set  cache  [7]  *-  k  and  char  [7)  *-  c.  In  other  cases  the  algorithm  should  set 
cache  [7']  *-  0.  Note  that  the  char  array  is  altered  by  this  algorithm,  but  only  in  G  commands. 

Our  algorithm  works  with  two  pointers  p  and  q,  where  1  <P<9<«  +  1.  Printer  p 
represents  the  current  position  where  wc  arc  defining  the  traee;  we  shall  say  that  the  trace  has  been 
defined  “up  to  time  p,”  thinking  of  a  clock  that  advances  when  p  increases.  Pointer  q  looks  ahead 
to  the  first  L  command  that  locks  a  character  not  in  the  cache  at  time  p;  if  no  such  commands 


exist,  we  have  q  —  n  +  1.  For  each  character  i  there  are  two  values 


f  0,  if  *  is 
sloth]  =  [  . 

I  the  cach< 


not  present  in  the  cache  at  time  p; 
cache  position  of  t,  otherwise. 
usage  [t]  =  the  number  of  L[»]  instructions  before  command  q  minus  the 
number  of  l/[»]  instructions  before  command  p. 


For  each  cache  position  k  <  s  we  will  have 


(.  0,  if  position  k  is  empty. 

Suppose  that  character  t  appears  in  different  “lock”  commands,  numbered  jn  <  j',a  < 
•••  <  jir..  A  preliminary  pass  over  the  char  array  suffices  to  Pill  two  auxiliary  arrays  first |i]  and 
next\j],  for  1  <  t  <  m  and  1  <  j  <  n,  so  that 


first[i\  =  jn  ,  next [;»,]  =  jn  ,  . . .  ,  next[ii,4]  =  n  +  1 . 

If  r*  =  0,  we  can  set  first [i]  =  n  +  1,  although  this  value  won’t  be  looked  at  so  it  really  doesn’t 
matter. 

Initially  p  =  q  =  t,  usage [*]  =  */ot[t]  =  0  for  1  <  i  <  m,  and  content* (fc)  =  0,  for 
1  <  h  <  s.  The  initial  value  of  first  [*']  will  be  jn  as  stated  above;  but  as  the  algorithm  progresses, 
,/ir*f{t]  will  be  updated  so  that  it  is  the  smallest  element  >  q  of  the  set  {i,i ,  i«j,  •  •  • ,  jiTi }  •  For 
convenience,  we  also  set  first [0]  =  n+ 1  and  uea;e[0]  —  0,  so  that  0  is  essentially  a  character  that 
never  appears. 

One  more  thing  completes  this  family  of  data  structures:  There  is  a  priority  queue  Q  of  all 
cache  positions  k  such  that  usage  [contents  [jfej]  =0;  these  positions  are  ordered  by  first  [contents  [*]]. 
Initially  Q  contains  all  positions  {1, . . . , «}  in  arbitrary  order.  Any  suitable  scheme  for  implement¬ 
ing  a  priority  queue  can  be  used  for  Q\  if  s  is  small,  a  sorted  linear  list  will  be  adequate,  while  if 
s  is  large  a  method  that  requires  at  most  0(log  s)  steps  per  operation  might  be  most  appropriate. 
Note  that  Q  contains  all  cache  positions  whose  contents  will  be  unwedged  at  all  times  between  p 
and  q  inclusive,  sorted  in  order  of  the  first  time  they  will  be  locked  after  time  q. 

The  algorithm  proceeds  by  advancing  p  one  step  at  a  time,  first  moving  q  as  far  as  it  can 
ahead  of  p: 

while  p  <  n  do 

begin  integer  i;  comment  bring  this  character  into  the  cache  next; 

[move  q  forward  until  reaching  L(i)  with  i  not  present); 

[process  command  p,  attempting  to  bring  in  t); 
p*-p  +  l; 

end. 

The  subalgorithm  that  moves  q  forward  will  set  t  to  the  character  that  should  be  brought  into  the 
cache  next;  this  is  the  character  not  present  at  time  p  that  is  going  to  be  needed  soonest.  If  no 
such  characters  exist,  we  will  have  q  *  n  +  1  and  *  «  0; 

* 


(move  9  forward  until  reaching  L(t)  with  t  not  present)  = 
begin  t  «—  0; 

while  9  <  n  and  t  =  0  do 

if  op\q\  'V  then  q  *-  q  +  1 
else  begin  *  «—  char\q ]; 
if  afot  [a]  >  0  then 

begin  /wstfi]  *—  next[q); 

if  usage  [t]  =  0  then  delete  slot [»']  from  Q; 

usage  [t]  =  usage  [»']  +  1; 

q  +-  q  +  1;  i  0; 

end; 

end; 

end; 

When  deleting  slot  [»']  from  Q,  it  may  help  to  know  that  slot  [*)  is  at  the  rear  of  Q;  i  is  Ihe  character 
that  would  currently  be  chosen  last  for  replacement  in  the  cache  on  the  basis  of  priority  since  it 
has  the  minimal  value  of  /irst  [contents  [t]]. 

The  processing  of  command  p  has  two  main  components,  depending  on  whether  the  command 
is  for  unlocking  or  bringing  in  a  character: 

(process  command  p,  attempting  to  bring  in  i)  = 

begin  cache  [p]  0;  comment  this  value  may  be  changed  later; 

if  op\p)  =  ‘U’  then  (unlock  char [p] ) 

else  if  *  >  0  and  (op [p]  =  'G'  or  p  =  q)  then  (try  to  bring  in  i  and  advance  9) 
end. 

The  first  of  these  is  a  simple  update  to  the  data  structures: 

(unlock  char\p\)  = 

begin  integer  j ;  comment  unlock  this  character; 

j  *-  char{p]; 

usage\j]  *-  usage  [j]  —  1; 

if  usage  \j\  =  0  then  insert  slot  (7]  into  Q  with  key  first\j]; 
end. 

The  other  operation  is  the  most  interesting: 

(try  to  bring  in  i  and  advance  9)  = 
if  Q  is  empty  then 

begin  if  p  —  9  then  report  overflow  error; 
end 

else  begin  integer  1;  comment  change  this  cache  position; 
delete  k  from  Q  with  maximum  /irst  [contents  [fc)j; 
cache  [pj  4-  fc;  char[p j  4-  *; 

slot  (contents  [h)J 4-  0;  slot[«]  4-  Jfc;  contents  [k]  4-  »; 
first\i\  4-  next [9];  use #*(»)  ♦-  1;  9  «-  9  + 

end; 


Note  that  if  p  =  q,  we  have  op\p]  =  ‘ L '  and  a  fault  has  occurred.  An  overflow  error  is  detected  ir 
p  —  q  and  Q  is  empty,  since  this  means  that  the  pth  command  is  trying  to  lock  some  character 
not  in  the  cache,  while  s  other  characters  are  already  wedged. 

It  is  straightforward  to  verify  that  the  operations  preserve  the  invariant  relations  we  have 
stated  for  the  data  structures,  and  therefore  that  an  optimum  strategy  5  is  being  found. 

Note  that  the  running  time  of  this  implementation  is  of  order  m  +  nlogs.  If  a  lot  of  G 
commands  are  present,  the  pointer  q  tends  to  be  quite  far  ahead  of  p  so  that  comparatively  few 
characters  in  the  cache  will  have  zero  usage;  thus  Q  will  not  contain  many  entries,  and  t'  running 
time  will  be  essentially  linear.  Thus,  additional  G  commands  will  make  the  algorithm  er,  even 
though  they  cause  it  to  find  the  optimum  over  a  larger  space  of  possible  strategies 

5.  Refinements  to  the  implementation. 

The  algorithm  of  Section  4  can  be  modified  in  various  ways  to  improve  its  efficiei.^  and  to 
take  account  of  practical  constraints. 

In  the  first  place,  the  running  time  will  be  improved  if  we  realise  that  p  usually  increases 
several  times  before  q  moves.  If  t  >  0,  so  that  op [4]  =  ‘L’  and  char  [9]  =  i  needs  to  be  brought 
in,  pointer  q  will  stand  still  until  the  code  (try  to  bring  in  i  and  advance  q)  is  actually  executed. 
Therefore  the  main  loop  of  the  program  can  be  reorganized  with  a  loop  on  q  followed  by  a  loop 
on  p  followed  by  an  operation  that  increases  both  p  and  9. 

In  the  second  place,  the  fact  that  n  is  large  means  that  it  is  undesirable  to  have  a  separate 
array  next]}]  for  i  <  j  <  »;  this  additional  array  limits  the  number  of  commands  that  can  be 
accommodated.  By  looking  at  the  way  this  .algorithm  uses  next,  we  can  sec  that  the  next  and 
char  arrays  can  be  overlapped  at  the  expense  of  a  (shorter)  array  second] j\  for  1  <  j  <  m. 
The  new  conventions  arc  as  follows,  if  the  “lock”  commands  following  time  q  for  character  i  are 
jil  <  •••  <  ja¬ 
il  n  —  0:  first  [»j  =  n  +  1,  seeond]i]  —  undefined. 

If  u  =  1:  /»rsf[t]  =  jii,  second [»]  =  n  +  1,  char  (j.i]  =  ». 

If  u  =  2:  first]*]  =  ju,  second]*]  =  ju,  char  (/«)  =  t,  char  [j^a]  =  «  +  1- 

If  U  >  3:  first [*)  s=  ju,  second [t]  =  J,a,  ehor(j,i)  =  char  (7^)  =  ji3, 

ehttr  [7  nu-i)]  —  jirt,  char  [7^]  =  n  +  1. 

The  operation  ‘first  [1)  4-  next  [9)’,  which  appears  twice  in  the  algorithm  of  Section  4  at  times  when 
r,  >  0,  is  now  changed  to  the  following  code: 
begin  integer  7; 
j  4-  second (t];  first]*]  —  7; 
if  j  <  »  then 

begin  second[t]  4-  char]j\\  char (7]  4-  i; 

In  the  third  place,  we  must  face  the  fact  that  jobs  generally  have  more  commands  than  could 
possibly  be  held  in  our  computer's  memory.  Ualher  than  having  the  algorithm  read  in  an  entire 


job,  figure  out  the  cache  trace  and  put  it  into  the  cache  array,  we  will  instead  regard  the  cache 
allocation  algorithm  as  a  coroutine  that  docs  the  caching  “on  line”  as  it  reads  the  commands.  In 
other  words,  if  we  can  store  only  no  commands  in  memory  at  once,  we  would  like  to  have  an 
algorithm  that  will  have  read  no  commands  ahead  of  the  one  it  is  actually  implementing  at  any 
given  time.  Thus,  when  the  coroutine  is  called  on  to  provide  the  value  of  cache[x\,  it  has  elements 
i  through  (x  +  no  —  1)  of  the  char  and  op  arrays  in  cyclic  buffers  in  memory.  The  coroutine  figures 
out  what  to  do  for  step  x,  and  then  it  reads  in  command  (x  +  no),  over-writing  op[x]  and  char\x\. 

When  lookahead  is  limited  to  no  future  commands,  we  might  not  discover  a  truly  optimum 
trace.  But  the  only  errors  we  make  would  be  to  remove  certain  items  from  the  cache  in  a  different 
order  when  those  items  are  not  used  at  all  during  the  next  no  steps.  If  no  is  large  enough  compared 
to  the  cache  size,  it  is  highly  likely  that  all  such  items  will  leave  the  cache  anyway,  even  in  an 
optimal  trace;  so  a  limited- lookahead  method  will  usually  be  no  worse  than  the  optimum.  Indeed, 
our  proof  or  optimality  in  Section  3  shows  that  a  variety  of  strategies  will  usually  perform  no  worse 
than  strategy  S. 

Implementation  of  the  coroutine  philosophy  means  that  we  need  to  update  the  first,  second, 
and  char  arrays  on-line  instead  of  assuming  that  they  have  been  initialized  by  a  preliminary  pass 
over  all  the  commands.  For  this  purpose  we  need  another  array  Iost[i]  for  1  <  i  <  m,  containing 
the  value  of  j,r<,  if  rt-  >  0;  we  leave  last  [ij  undefined  if  rt-  —  0.  Furthermore  some  other  sentinel 
value  must  be  used  instead  of  n  +  1  in  the  first  and  second  arrays,  since  we  don’t  know  what  n 
is.  We  shall  use  0;  the  teat  ' j  <  n’  above  should  therefore  be  changed  to  '  j  >  O’. 

The  algorithm  now  starts  by  filling  up  the  op  and  char  arrays  with  the  first  n0  commands  in 
the  job,  the  first,  second  and  last  arrays  are  set  up  to  reflect  these  commands,  and  p  and  q  are 
set  to  1.  The  entire  data  structure  must  be  kept  up  to  date  as  p  and  q  change.  For  instance,  as 
p  is  incremented  to  2,  the  algorithm  should  put  command  number  no  +  1  into  op[l]  and  c/iar[l], 
and  update  the  first,  second,  and  last  arrays  to  reflect  this  new  command.  Thus,  the  statement 
‘p  «—  p  +  1*  is  replaced  by: 

(advance  p)  = 

begin  (op[p],  char[p\)  *—  next  command  in  the  job; 

if  opjpj  =  'L'  then 

begin  integer  i;  i  *-  char  [p] ; 
if  /IrafJjj  =  0  then 

begin  /iraf[»J  «-  p;  second  [i] «—  0; 

(if  aiotjt]  €  Q  then  change  its  key); 
end 

else  begin  char[p\  «—  0; 

if  second  [i]  —  0  then  second  [»}  *—  p 
else  char[last[i]]  «—  p; 
end; 
foat(i)  4-  p; 
end; 

if  p  =  no  then  p  ♦—  1  else  p  ♦—  p  +  1 ; 

end. 
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The  statements  'q  «-  q  +  1*  arc  changed,  to: 
if  4  =  no  then  q  *-  1  else  q  «-  q  +  1; 

A  few  other  changes  to  the  code  arc  required  to  keep  q  from  incrementing  when  it  gets  no  commands 
ahead  of  p. 

A  “dead”  character  is  one  that,  as  far  a  we  can  tell  from  our  limited  lookahead,  will  never 
again  be  used  in  the  job.  Thus,  character  i  is  dead  if  and  only  if  usage (t]  =  0  and  =  0. 

It  is  convenient  to  split  Q  into  two  separate  parts:  Qq,  which  is  simply  an  unordered  set  of  all 
cache  positions  which  arc  empty  or  contain  dead  characters,  and  the  remaining  part  Q i,  which 
is  a  priority  queue  ordered  by  the  nonzero  key  values  first  [contents  [&]].  These  key  values  are  to 
be  “circularly  ordered”  in  the  sense  that  we  regard  x  >  y  it  x  <  p  <  q  <  y,  since  x  is  one  lap 
ahead  of  y  in  such  a  case.  Note  that  the  operation  {if  sfof[t]  €  Q  then  change  its  key)  simply 
removes  slot  [a]  from  Qo  and  enters  it  into  Q\  with  key  p,  which  will  be  higher  than  any  other  key 
currently  in  Q i.  The  elements  of  Qo  are  all  regarded  as  having  higher  keys  than  those  of  Q\. 

It  is  a  simple  matter  to  (ill  in  all  the  remaining  details:  to  take  care  of  shutting  down  the 
input  operations  when  all  commands  have  been  read  and  to  terminate  the  coroutine  when  all  of 
the  cache  commands  have  been  implemented. 


6.  Empirical  testa. 

The  authors  have  used  these  procedures  to  drive  an  Alphatype  CRS  phototypesetter,  producing 
such  technical  books  as  (4].  In  this  application  the  characters  in  the  cache  have  variable  size,  so 
the  actual  cache  storage  is  allocated  dynamically.  When  a  new  character  is  brought  into  the  cache, 
there  might  already  be  room,  but  on  the  other  hand,  it  might  be  necessary  to  remove  several 
other  characters  before  a  hole  appears  that  is  large  enough  to  accommodate  an  especially  large 
newcomer.  The  number  of  G  commands  at  the  end  of  a  line  is  not  fixed,  because  it  depends  on 
the  sizes  of  characters  that  are  actually  brought  in. 

In  other  words,  the  theoretical  model  studied  earlier  in  this  paper  was  a  rather  drastic 
simplification  of  the  actual  problem  that  had  arisen  in  practice.  As  usual.  But  (as  usual)  the 
theoretical  considerations  provided  valuable  guidelines  for  a  practical  implementation,  and  by  using 
an  algorithm  that  is  optimal  or  near-optimal  under  the  simplifying  assumptions,  the  authors  were 
able  to  achieve  quite  satisfactory  results  even  though  those  assumptions  were  violated. 

Indeed,  it  would  almost  surely  be  unfeasible  to  develop  an  optimum  strategy  that  takes  account 
of  all  the  details  of  the  actual  application,  since  the  problem  of  optimum  dynamic  storage  allocation 
is  already  NP-complcte  before  we  add  the  extra  complexities  of  cache  management.  (See  [3], 
problem  SR2.)  Instead  of  worrying  about  special  schemes  for  dynamic  allocation,  the  authors 
found  that  it  was  sufficient  to  replace  unwedged  characters  simply  on  the  basis  of  their  priority, 
without  regard  to  their  size  or  to  the  priorities  or  sizes  of  their  neighbors. 

Figure  1  shows  a  sample  text  that  was  subjected  to  a  variety  of  experiments  discussed  below. 
This  text  had  been  used  to  debug  the  T^jX  typesetting  system  in  1978,  and  it  also  provided  the 
style  pages  in  the  design  of  [4];  thus  it  represents  a  wide  variety  of  different  things  that  happen  in 
a  700-page  book,  compressed  into  about  four  pages.  It  involves  the  typesetting  of  5211  characters, 
of  which  576  arc  distinct  when  size  variations  arc  taken  into  account. 
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The  task  of  driving  the  authors’  typesetting  equipment  can  be  described  in  terms  of  the 
abstract  model  of  Section  2  as  the  problem  of  implementing  a  sequence  of  commands  having  the 
following  general  form: 

Lock  all  characters  used  on  line  fc; 

Tell  the  typesetter  to  start  setting  line  k; 

If  time  permits,  issue  G  commands  to  bring  in  future  characters; 

Unlock  all  characters  used  on  line  (k  —  1). 

We  do  this  for  k  =  1,2,...,  except  that  the  pattern  changes  in  special  cases.  The  term  “line” 
means  a  sequence  of  characters  that  arc  to  be  typeset  at  the  same  baseline;  thus,  a  complex 
mathematical  formula  might  actually  occupy  many  lines.  There  is  usually  time  to  preload  future 
characters  into  the  cache,  because  the  time  to  transmit  the  information  about  what  to  set  on  line  k 
is  usually  less  than  the  lime  for  the  actual  typesetting  of  that  line. 

Note  that  there  are  generally  two  consecutive  lines  wedged  in  the  cache  at  once,  since  line  (k—  1) 
isn’t  unlocked  until  after  line  k  has  been  locked;  this  is  due  to  buffering  inside  the  typesetting 
machine.  In  emergency  situations,  when  the  ordinary  policy  would  overload  the  cache,  line  (Jfc  —  1) 
will  be  unwedged  sooner  and  the  controlling  process  will  pause  to  make  sure  that  the  buffer  is  clear; 
the  cache  will  also  be  repacked  at  such  times  in  order  to  make  ail  of  the  available  memory  appear  in 
consecutive  locations.  Also,  if  the  typesetter  is  still  busy  doing  line  k  when  the  controlling  process 
begins  to  tell  it  about  typesetting  line  ( k  +  1),  the  typesetter  will  stop  taking  commands  until  it 
is  through  with  line  k .  Note  that  this  allows  characters  from  line  (k  +  l)’s  G  commands  to  be 
brought  in  while  line  k  is  still  being  typeset,  -fane  {k  +  l)’s  L  commands  that  cause  faults  will  not 
entirely  overlap,  since  the  G  commands  should  account  Tor  most  or  the  time  that  the  typesetter 
spends  on  line  k. 

Special  actions  occur  at  the  beginning  of  a  page:  If  the  Ijlm  has  to  move  comparatively 
far  in  order  to  be  in  the  proper  position  to  start  the  new  page,  there  is  extra  lime  to  preload 
font  information,  hence  the  controlling  process  issues  additional  G  commands.  In  particular,  the 
characters  for  the  first  lines  of  the  first  page  will  generally  have  been  brought  into  the  cache  by 
the  time  the  typesetter  is  positioned  at  the  top  baseline. 

Several  dosen  experiments  were  performed  on  Figure  1  in  order  to  get  some  idea  as  to  how 
the  algorithm  performs  under  various  conditions.  The  cache  sise  was  varied  so  that  it  would  be 
able  to  hold  approximately  50,  75,  100,  125,  or  150  characters;  we  shall  refer  to  these  sites  as 
C50,  C75,  . . . ,  C150,  respectively.  The  speed  at  which  font  information  could  be  transmitted  was 
varied  so  there  was  free  time  to  send  cither  an  average  of  six  new  characters  per  line  (i.c.,  about 
six  G  instructions  after  each  line),  or  about  4.5  new  characters  per  line,  or  no  such  characters; 
in  the  latter  case,  no  G  commands  arc  given,  so  the  algorithm  must  minimise  the  total  number 
of  characters  transmitted.  We  shall  refer  to  these  transmission  speeds  as  06,  G4.5,  and  GO.  The 
algorithm  was  also  run  in  four  modes:  (1)  with  full  lookahead;  (ii)  with  internal  memory  cut  back 
so  that  only  about  12  lines  of  data  could  be  accommodated  at  once;  (ili)  with  internal  memory 
cut  back  to  only  about  6  lines;  and  (iv)  with  full  lookahead  but  with  the  priority  queue  decisions 
reversed  so  that  the  worst  possible  cache  replacements  were  made  whenever  the  algorithm  had  to 
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take  something  from  the  cache.  These  four  lookahead  modes  will  be  called  Loo,  L12,  L6,  and  IX), 
respectively.  Five  cache  sites,  three  speeds,  and  four  lookahead  modes  make  for  sixty  combinations, 
and  so  sixty  experiments  were  performed  and  the  resulting  numbers  of  faults  are  shown  in  Table  A. 

Table  A 

FAULTS  THAT  OCCUR  WHEN  TYPESETTING  FIGURE  1 


GO  G4.S  G6 


LO 

L6 

L12 

Loo 

L0 

L6 

L12 

Loo 

L0 

L6 

L12 

Loo 

C50 

1881 

1060 

1040 

1037 

572 

268 

268 

254 

378 

198* 

204 

197  C50 

C75 

1863 

060 

856 

834 

380 

01 

76 

69 

146 

35 

31* 

32  C75 

C100 

1854 

054 

780 

752 

353* 

70* 

30 

27 

110 

20* 

0* 

3  C100 

C125 

1821 

041 

786 

609 

381 

83 

26 

12 

22 

22 

0 

0  C125 

Cl  50 

1810 

017 

770 

614 

356 

66 

26 

9 

0* 

22 

0 

0  C150 

(Asterisks  denote  “anomalous”  values  that  are  surpringly  low.) 


These  results  are  quite  encouraging.  Consider  first  the  GO  case,  when  no  “frccloading”  is 
done:  At  least  576  faults  must  occur,  since  each  distinct  character  must  be  brought  in  at  least 
once,  and  the  table  shows  that  a  caching  strategy  with  lookahead  is  able  to  make  sure  that  only  a 
few  characters  need  to  be  brought  in  twice.  The  number  of  faults  under  G4.5  is  substantially  less, 
even  for  the  unusually  complicated  text  of  Figure  1;  and  with  G6  and  a  moderately  large  cache 
the  faults  disappear  entirely. 

The  starred  entries  in  Table  A  show  interesting  anomalies  where  a  lucky  combination  of 
circumstances  led  to  fewer  faults  than  would  be  expected.  Consider,  for  example,  the  cases  with 
G4.5  and  L6  or  LO,  where  the  cache  site  C100  turned  out  to  be  slightly  better  than  C125.  The 
reason  was  that  these  inherently  nonoptimal  strategies  made  better  guesses  in  the  C100  case. 
Another  interesting  example  is  the  case  C6  and  C150,  where  the  supposedly  pessimal  strategy 
LO  actually  did  better  than  L6.  The  reason  here  is  that  LO  only  pcssirnizes  the  choice  of  cache 
replacements.  The  other  part  of  our  algorithm,  which  looks  ahead  to  find  the  next  candidate  for 
G  bringing  in,  remains  optimum;  and  when  there  are  enough  G’s,  this  part  of  the  algorithm  is 
strong  enough  to  make  the  replacement  strategy  immaterial.  On  the  other  hand  the  L6  restriction 
curtails  the  effectiveness  of  the  G  lookahead  as  well  as  the  replacement  lookahead,  so  L6  can  come 
out  worse.  The  22  faults  occurred  at  the  beginning  of  Figure  l’s  page  3,  where  a  conversion  from 
nine-point  to  ten-point  type  takes  place;  L6  wasn’t  prepared  for  so  many  changes  all  at  once. 

The  most  interesting  anomaly  arose  in  the  case  C100  and  G6,  when  the  suboptimal  strategy 
L12  actually  turned  out  to  be  better  than  the  supposedly  optimal  Loot  A  careful  examination  of 
what  happened  shows  that  this  was  a  case  of  good  luck  for  L12  and  bad  luck  for  Loo.  It  all  started 
when  the  typesetting  was  going  along  routinely,  about  ten  lines,  from  the  bottom  of  page  1;  both 
Loo  and  LI  2  were  doing  approximately  the  same  thing,  but  with  minor  variations  so  that  their 
dynamic  storage  allocation  patterns  in  the  font  cache  were  quite  different.  Both  strategics  had 
succeeded  in  looking  rather  far  ahead,  and  they  were  beginning  to  bring  in  the  eight-point  upper¬ 
case  letters  needed  for  the  caption  at  the  top  of  page  2.  But  when  the  “optimal"  Loo  strategy  had 


successfully  brought  in  the  eight-point  ‘O'  and  ‘L’,  its  cache  had  no  free  blocks  big  enough  to  bring 
in  the  ‘S’.  The  restricted  L12  strategy,  on  the  other  hand,  had  a  fortuitous  memory  configuration 
that  allowed  it  to  bring  in  not  only  the  ‘S’  but  also  the  ‘I’  and  ‘N’.  This  put  L12  three  characters 
ahead  of  Loo,  and  it  retained  a  three-character  advantage  all  the  way  through  page  2  and  the 
beginning  of  page  3,  where  comparatively  rapid  font  changes  caused  the  lookahead  to  evaporate. 
Finally  L12’s  lead  manifested  itself  on  the  line  before  (1)  on  page  3;  three  faults  occurred  when 
Loo  had  to  bring  in  ‘W’  and  the  two  pairs  of  quotation  marks. 

Note  that  L12  was  almost  never  a  great  deal  worse  than  Loo,  in  any  of  the  cases,  so  it  appears 
that  a  restricted  lookahead  still  makes  a  satisfactory  approximation  to  optimal  behavior.  In  the 
authors'  application  it  turns  out  that  there  is  enough  core  memory  to  look  about  2500  lines  ahead; 
experiments  show,  however,  that  L50  is  essentially  equivalent  to  Loo,  thus  the  storage  requirements 
can  be  reduced  greatly  from  what  we  originally  thought  would  be  necessary. 

Figure  2  shows  a  detailed  trace  of  what  went  on  in  the  experiment  for  case  (G4.5,  Loo,  CI25). 
The  horixontal  axis  separates  the  834  characters  that  were  brought  in  during  the  time  Figure  1  was 
being  typeset;  all  but  12  of  these  were  brought  in  during  G  commands,  while  the  remaining  12  were 
faults.  The  vertical  axis  represents  the  314  lines  in  Figure  1.  The  graph  shows  two  xig-sag  paths, 
where  the  upper  one  represents  each  character’s  first  use.  Thus,  the  upper  path  is  far  above  the 
lower  path  when  characters  arc  being  preloaded  many  lines  ahead,  while  the  two  paths  touch  each 
other  when  a  fault  has  occurred.  The  lower  path  has  a  somewhat  erratic  behavior:  occasionally  we 
find  a  horixontal  segment  on  that  path,  representing  a  line  that  introduces  many  new  characters. 
(The  worst  cases  are  the  line  following  ‘EXERCISES — Special  set’  and  the  line  beginning  ‘3.3.3.3. 
This  subsection  doesn’t  exist’,  both  of  which  required  31  new  characters  to  be  preloaded  in  order 
to  avoid  faults.)  The  upper  path,  on  the  other  hand,  is  more  regular,  because  there  is  roughly  the 
same  amount  of  time  for  prcloading  characters  on  each  line.  Variations  in  the  upper  path  occur 
when  the  characters  to  be  brought  in  arc  especially  large  or  small,  or  when  the  line  being  typeset 
is  short  (as  at  the  end  of  a  paragraph),  or  when  the  baselines  are  far  apart;  but  these  changes  are 
comparatively  minor. 

Sometimes  the  cache  is  full,  so  that  the  lookahead  procedure  stops  and  the  current  G  com¬ 
mands  are  not  used.  This  is  indicated  in  Figure  2  by  the  symbol  ‘|— ’  on  the  upper  path;  the  first 
such  incidents  occur  near  the  bottom  of  page  2  in  Figure  1,  and  a  more  significant  stoppage  occurs 
during  the  big  displayed  equations  near  the  bottom  of  page  3. 

Before  developing  the  algorithms  described  above,  the  authors  did  a  hand  simulation  on 
some  sample  text  using  the  assumptions  (C100,  Loo,  G6),  since  these  parameters  appeared  to 
be  appropriate  for  the  typesetting  equipment  that  Stanford  planned  to  acquire.  The  success  of 
caching  with  these  parameters,  in  spite  of  the  multiplicity  of  fonts  needed  to  typeset  difficult 
technical  material,  encouraged  us  to  proceed  further.  Two  years  later,  after  the  hardware  and 
software  were  put  Into  production,  we  found  that  G4.5  was  more  appropriate  than  G0,  because 
time-sharing  interfered  with  transmissions  to  the  typesetter;  however,  this  was  compensated  by 
saving  space  in  the  typesetter  software  so  that  C125  was  more  representative  of  the  actual  cache 
rise.  In  fact,  the  new  Alphatype  model  400  arrived  with  additional  cache  memory,  so  that  our 
current  software  corresponds  to  C155  and  faults  hardly  ever  occur. 

Note  that  the  strategy  of  Section  3  does  not  minimise  the  number  of  times  a  character  is 
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brought  into  the  cache;  it  only  minimiles  the  number  of  faults.  For  example,  consider  a  cache  of 
sise  2  and  the  job 


L(  1)  L(2)  1/(1)  G  1/(2)  G  1(3)  L(  1)  U(l)  1/(3). 

Strategy  S  will  bring  in  3  at  the  first  G,  then  bring  in  1  at  the  second;  the  alternative  of  passing 
on  the  first  G  and  bringing  in  3  on  the  second  would  be  preferable  if  we  were  trying  to  minimise 
the  number  of  brings. 

Bach  time  a  character  is  brought  into  the  cache  and  docs  not  cause  a  fault,  typesetting  is  not 
slowed  down,  but  the  amount  of  information  that  must  be  sent  to  the  typesetter  docs  increase,  so 
it  is  desirable  to  try  to  minimise  the  number  of  times  characters  are  brought  into  the  cache.  The 
algorithm  of  Sections  4  and  5  can  be  modified  to  "pass”  a  G  if  there  are  no  dead  characters  in  the 
cache  (i.e.,  if  Qq  is  empty),  provided  that  the  lookahead  pointer  q  is  sufficiently  far  from  p  that  it 
is  reasonably  safe  to  assume  we  will  be  able  to  avoid  faults  by  acting  on  future  G' s. 

For  example,  suppose  lookahead  stops  whenever  it  would  require  the  replacement  of  a  non¬ 
dead  character,  provided  that  the  algorithm  has  looked  ahead  so  far  that  the  next  character  to  be 
brought  in  is  16  or  more  lines  away  from  the  current  line  being  typeset.  Let  us  call  this  variant 
U16.  Then  the  algorithm  may  well  be  able  to  avoid  rashly  replacing  characters  that  are  not  dead, 
by  holding  back  until  a  character  becomes  dead,  without  seriously  risking  future  faults.  Figure  2 
shows  834  characters  brought  in  when  the  parameters  are  (Loo,  G4.5,  C125);  but  if  the  distance 
between  the  upper  path  and  lower  path  were  constrained  to  be  no  more  than  about  16  or  so,  it  is 
plausible  to  believe  that  we  would  end  up  bringing  in  characters  fewer  times,  and  we  might  even 
be  able  to  approach  the  optimum  of  699  achieved  in  the  case  GO.  The  following  data  show  what 
happens  for  Loo,  G4.5,  and  C125: 


Uoo 

U24 

U16 

U8 

UO 

faults 

12 

24 

24 

26 

105 

brings 

834 

831 

795 

761 

725 

With  U16,  there  arc  39  fewer  characters  brought  into  the  cache,  at  a  cost  of  12  faults. 

But  Figure  1  is  not  a  typical  example.  Therefore  further  tests  were  made  on  “real”  data.  The 
text  of  Section  3.5  of  (4)  is  representative  of  the  difh.ultiea  of  a  normal  mathematical  paper,  so 
it  serves  as  a  good  indication  what  we  can  usually  expect.  This  second  test  case,  which  amounts 
to  28  typeset  pages,  involves  the  setting  of  57912  characters,  660  of  which  are  distinct.  When 
the  algorithm  was  applied  with  parameters  (Loo,  G4.5,  C125,  Uoo)  there  were  only  17  faults,  and 
these  all  occurred  near  the  very  beginning.  The  total  number  of  characters  brought  in  to  do  the 
whole  job  was  2745;  and  with  the  U16  heuristic,  this  dropped  to  2131,  while  the  number  of  faults 
remained  at  17. 

Several  other  experiments  were  made  in  the  3.6  file,  holding  all  but  one  of  the  settings  (Loo, 
G4.5,  C125)  fixed.  When  Loo  was  changed  to  L12,  there  still  were  only  17  faults;  restricting  further 
to  L6  increased  them  slightly  to  44.  And  when  Loo  was  changed  to  the  “pessimising”  L0,  the  result 
was  17  again!  Thus,  the  lookahead  process  appears  to  be  powerful  enough  to  achieve  optimality 
without  the  refinement  of  the  priority  queue,  when  we  consider  typical  data,  provided  that  the  G 
speed  and  the  esc  he  sise  are  suitably  large. 


As  expected,  the  17  faults  vanished  at  speed  G6.  Reducing  the  speed  to  G3  increased  the 
number  of  faults  to  65;  these  occurred  only  at  the  beginning  and  at  the  switch  to  nine-point  type 
for  the  exercises.  With  speed  G1.5  there  were  248  faults,  and  with  speed  G0.1  there  were  1144; 
speed  GO  gave  1560.  (This  compares  with 

G6  G4.5  G3  G1.5  G0.1  GO 

0  12  112  333  613  609 

in  the  case  of  Figure  1.) 

Increasing  the  cache  site  to  C150  did  not  reduce  the  number  of  faults  below  17.  With  a  setting 
of  raze  C100  there  were  26  faults,  while  C75  gave  201.  Sise  C50  was  not  quite  large  enough  to  hold 
all  of  the  characters  wedged  in  one  of  the  nine-point  lines;  C52  gave  1406  faults. 

We  can  summarize  these  requirements  by  saying  that  typical  technical  text  can  be  typeset 
with  negligibly  few  faults  provided  that  the  algorithm  of  this  paper  is  used  in  connection  with  the 
following  resources: 

i)  A  cache  in  the  typesetter  capable  of  holding  about  125  character  shape  descriptions; 

ii)  Time  to  preload  about  4  characters  per  line  without  slowing  down  the  typesetting  process; 

iii)  Enough  memory  in  the  host  computer  to  look  ahead  about  12  lines  (i.e.,  about  750  characters) 
in  the  text  to  be  typeset. 
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